CMD: Self-supervised 3D Action Representation Learning with Cross-Modal Mutual Distillation

In 3D action recognition, there exists rich complementary information between skeleton modalities. Nevertheless, how to model and utilize this remains a challenging problem for self-supervised representation learning. work, we formulate the cross-modal interaction as bidirectional knowledge distillation problem. Different from classic solutions that transfer of fixed pre-trained teacher student, in is continuously updated bidirectionally distilled To end, propose new Cross-modal Mutual Distillation (CMD) framework with following designs. On one hand, neighboring similarity distribution introduced learned each modality, where relational naturally suitable contrastive frameworks. other asymmetrical configurations are used student stabilize process high-confidence By derivation, find positive mining previous works can be regarded degenerated version our CMD. We perform extensive experiments on NTU RGB+D 60, 120, PKU-MMD II datasets. Our approach outperforms existing methods sets series records. The code available at: https://github.com/maoyunyao/CMD .